How Much Does Network Contention Affect Distributed Shared Memory Performance?

Most of recent research on distributed shared memory (DSM) systems have focused on either careful design of node controllers or cache coherence protocols. While evaluating these designs, simplified models of networks (constant latency or average latency based on the network size) are typically used. Such models completely ignore network contention. To help network designers to design better networks for DSM systems, in this paper, we focus on two goals: 1) to isolate and quantify the impact of network link contention and network interface contention on the overall performance of DSM applications and 2) to study the impact of critical architectural parameters on these two categories of network contention. We achieve these goals by evaluating a set of SPLASH2 benchmarks on a DSM simulator using three network models. For an 8 8 wormhole system, our results show that network contention can degrade performance up to 59.8%. Out of this, up to 7.2% is caused by network interface contention alone. The study indicates that network contention becomes dominant for DSM systems using small caches, wide cache line sizes, low degrees of associativity, high processing node speeds, high memory speeds, low network speeds, or small network link widths.